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Three-dimensional user interface for controlling a 
virtual reality graphics system by function selection 

5 Description 

The present invention generally relates to graphics 
systems for virtual reality (VR) applications and 
specifically relates to a graphical user interface for 
10 controlling such a VR graphics system by means of 
interactions with a function selection system that 
provides at least two functions and to a corresponding 
VR graphics system as claimed in the preambles of the 
respective independent claims. 

15 

A VR graphics system which is concerned in this case is 
evident from DE 101 25 07 5 Al, for example, and is used 
to generate and display a multiplicity of three- 
dimensional views which together represent a so-called 

20 "scene". In this case, such a scene is usually 
correspondingly visualized using the method (which is 
known per se) of stereoscopic projection onto a screen 
or the like. So-called immersive VR systems which form 
an intuitive man-machine (user) interface for the 

2 5 various areas of use (fig. 1) are widespread. Said 
graphics systems use a computer system to highly 
integrate the user into the visual simulation. This 
submersion of the user is referred to as "immersion" or 
an "immersive environment". 

30 

As a result of the fact that three-dimensional data or 
objects are displayed to scale and as a result of the 
likewise three-dimensional ability to interact, these 
data or objects can be assessed and experienced far 
35 better than is possible with standard visualization and 
interaction techniques, for example with a 2D monitor 
and a correspondingly two-dimensional graphical user 
interface. A large number of physical real models and 
prototypes may thus be replaced with virtual prototypes 
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in product development. A similar situation applies to 
planning tasks in the field of architecture, for 
example. Function prototypes may also be evaluated in a 
considerably more realistic manner in immersive 
5 environments than is possible with the standard 
methods . 

Such a visual VR simulation is controlled in a 
computer-aided manner using suitable input units 

10 (referred to below, for the purpose of generalization, 
as "interaction units" since their function clearly 
goes beyond pure data input) which interact with a user 
interface that can be temporarily inserted into the VR 
simulation. In addition to pushbuttons, the interaction 

15 units have a position sensor which interacts, via a 
cable or radio connection, with a position detection 
sensor system (which is provided in the VR graphics 
system) and can be used to continuously measure the 
spatial position and orientation of the interaction 

2 0 unit in order to carry out the interactions with the 

user interface on the basis of the physical movement, 
position and orientation of the interaction unit in the 
space . 

25 A corresponding graphical user interface is disclosed, 
for example, in DE 101 32 243 Al . The handheld 
cableless interaction unit described there is used for 
generating and transmitting the location, position 
and/or movement data (i.e. spatial position coordinates 

3 0 of the interaction unit) provided by a position sensor 

(already mentioned) and thus, in particular, for 
virtual three-dimensional navigation in an existing 
scene. Said position data comprise the six possible 
degrees of freedom of translation and rotation of the 
3 5 interaction unit and are evaluated in real time in a 
computer-aided manner in order to determine a movement 
or spatial trajectory of the interaction unit. 
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The graphical user interface described in DE 101 32 243 
Al comprises, in particular, a menu system which is 
likewise visualized in a three-dimensional 

(stereoscopic) manner, for example a spherical menu 
5 which can be controlled using translational and/or 
rotational movements of the interaction unit. In this 
case, functions or menu items are selected, for 
example, by means of a rotational movement (which is 
carried out by the user) of the interaction unit. 

10 

In the case of these user interfaces, it is desirable 
for the operation of said interactions for operating 
and controlling a function selection or menu system 
(which is concerned in this case) to be configured in 
15 an even simpler and more intuitive manner, particularly 
in the case of more complex function selection 
operations. At the same time, however, the highest 
possible degree of operational reliability and 
operating safety is also intended to be ensured. 

20 

The inventive graphical user interface for controlling 
a virtual reality (VR) graphics system (which is 
concerned in this case) by means of said interactions 
comprises a visual interaction element which 

25 functionally and visually comprises at least two 
subelements which interact with one another, each of 
these subelements providing a function selection having 
at least two respective functions. This at least two- 
part interaction element is preferably implemented in 

3 0 the form of a virtual three-dimensional menu system or 
function selection system. 

In particular, the at least two subelements are 
designed such that they can be moved in a virtual 
3 5 three-dimensional manner relative to one another by 
means of a physical three-dimensional movement of the 
interaction unit, said function or menu being selected 
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by means of the at least two subelements being moved 
relative to one another. 

In one preferred refinement, at least the first 
5 subelement of the visual interaction element that is 
inserted into the scene at least temporarily is 
displayed at an at least temporarily fixed position 
within the scene, at least the second subelement being 
able to be moved both functionally and visually in a 

10 virtual three-dimensional manner by means of a physical 
three-dimensional movement of the interaction unit 
relative to the first subelement - similar to the known 
"notch and bead sights" principle - in order to trigger 
a function by means of this relative movement between 

15 the at least two subelements. 

According to another refinement, this relative movement 
is effected, in the case of a translational 
displacement, in such a manner that the at least two 
20 subelements at least partially touch or overlap, which 
is likewise visualized in the scene, as a result of 
which said function or menu selection and thus, 
overall, operation and control of the user interface 
appear to be very intuitive and thus also user- 

2 5 friendly. 

In one particularly advantageous refinement, the 
proposed visual interaction element comprises three 
subelements, to be precise, in the case of a spherical 

3 0 menu, an inner sphere which is formed in one part, a 

spherical shell which is formed from at least two 
spherical shell segments and is arranged on the surface 
of the inner sphere and a ring which is arranged in the 
outer region of the sphere or spherical shell and 
35 comprises at least two ring segments. In this 
refinement, the inner sphere is used to represent an 
item of state information relating to the instantaneous 
state of the entire spherical menu, for example the 
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instantaneous position in a menu tree. That is to say 
said state information indicates, for example, whether 
the menu items which are represented by the spherical 
shell segments are a main menu or, for instance, a 
5 submenu that is hierarchically subordinate to the main 
menu. A function which is to be triggered using the 
outer ring is preferably activated, in this refinement, 
by means of the inner sphere making contact with, or 
overlapping, one of the at least two ring segments. 

10 

When operating such a spherical menu, the spherical 
shell segments can be correspondingly rotated about the 
inner sphere, by means of user-guided rotation of the 
interaction unit, in order to make it possible for 
15 different spherical shell segments to overlap the 
available ring segments, for example. In order to 
further simplify such control, another refinement 
provides an angle-dependent (for example in 30° steps) 
latching function which depends on the angle of 

2 0 rotation of the interaction unit, with the result that 

the spherical shell segments and the ring segments are 
always clearly opposite one another and ambiguous 
interactions between these segments are therefore 
virtually excluded . 

25 

In order to further increase the operating comfort, 
provision may additionally be made for a further 
relative displacement to be actively prevented as of a 
prescribable degree of partial overlap/ touching between 

3 0 the inner sphere and the ring. In addition to said 

rotation-dependent latching function, this also enables 
latching, which takes place in the event of 
trans lational movements of the inner sphere, along the 
possible displacement path of the sphere. 

35 

In order to render said translational movements of the 
inner sphere relative to the ring more intuitive and 
thus more user-friendly, the sphere element is 
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displaced relative to the ring element, in a further 
refinement, as if the inner sphere were connected to 
the individual ring segments via imaginary elastic 
bands or the like. This likewise ensures, in the manner 
5 of a latching function, that the translation of the 
inner sphere is also always led or is even forced to 
lead to a particular ring segment, and an adjacent ring 
element, for instance, cannot be driven inadvertently. 

10 The actions or functions which are triggered according 
to the invention by rotational and translational 
movements may be controlled and evaluated using 
empirically prescribable threshold values in such a 
manner that a physical three-dimensional translational 

15 or rotational movement (which is carried out by the 
user) of the interaction unit triggers a corresponding 
action or function only when the magnitude of the 
movement exceeds the respective threshold value. This 
makes it possible to more effectively prevent incorrect 

2 0 operation, for example an account of physical movements 
of the interaction unit which are effected 
inadvertently . 

The inventive user interface may also be visually 

2 5 displayed in animated form in such a manner that, when 

the respective movable subelement (for example the 
above-described inner sphere) is moved or in the event 
of the at least two subelements (for example the above- 
described inner sphere and the outer ring) 

3 0 touching/ overlapping, a change in the form or shape of 

at least one of these subelements occurs. 

Said functional sequences of the proposed user 
interface may also be assisted by means of at least one 
3 5 control element (pushbutton or the like) which is 
arranged on the interaction unit. By way of example, 
such a control element may be used to trigger not only 
the insertion of the visual interaction element into 
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the respective scene but also other functions, for 
example activation of the abovementioned 

touching/overlapping function etc. It goes without 
saying that, as an alternative to such a control 
5 element, the voice and/or gestures/facial expressions 
of the user may also be evaluated in a manner known per 
se. The abovementioned functions may thus be 
implemented, for example, by means of simple voice 
commands, for example, "open menu system", "activate 
10 overlapping function" or the like. 

In another particularly advantageous refinement, 
provision may be made for said touching/ overlapping 
function to additionally comprise logic (boolean) 

15 operations, i.e. when a subelement touches or overlaps 
a particular second subelement of the inventive user 
interface, a particular logic operation is carried out, 
a function, menu selection or the like, which is formed 
only by the respective logic combination, being carried 

2 0 out. This makes it possible for the inventive user 
interface to also be adapted to very complex functional 
sequences . 

As a result, the inventive graphical user interface 
2 5 thus affords the advantage that even complex 
interactions, for example over a plurality of function 
or menu levels, can be effected very intuitively, to be 
precise solely by means of said movement modes (in the 
six possible degrees of freedom as regards translation 
30 and rotation) of the interaction unit. Said 
overlapping/ touching function makes it possible, in 
particular, to rapidly and reliably change over 
between, for example, different subf unctions or 
submenus of a function selection or of a menu system. 

35 

In comparison with the prior art mentioned at the 
outset, the inventive user interface is therefore 
easier to handle and at the same time has a very high 
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level of operational reliability as regards possible 
operating faults caused by a user. Overall, virtual 
three-dimensional navigation is therefore considerably 
simplified as a result of different function/menu 
5 levels which are inserted into the scene, to be precise 
even without the use of a pointer which is frequently 
used in the prior art and is displayed in animated 
form. 

10 The invention can be used, with said advantages, both 
in VR graphics systems having cableless interaction 
units and in those having cable-bound interaction units 
which are preferably hand-guided by the user. As 
already stated, in addition to said use of a pushbutton 

15 that is arranged on the interaction unit, the possible 
user interactions may generally also be assisted in 
this case by acoustic or optical interactions, for 
example voice, gestures or the like. 

2 0 In addition, it goes without saying that, in the case 
of the user interface proposed, it is not important 
which of the two subelements is moved (i.e. translated 
or rotated) relative to which subelement and which of 
the subelements is respectively fixed and which is 

2 5 respectively movable in the preferred refinement. 

Instead of an above-described spherical menu, the 
invention may also be used with said advantages in a 
menu system which is of completely different graphical 

3 0 design if the menu system has at least two parts in the 

manner mentioned. Use in three-dimensional planar text 
menu systems or the like is thus also suitable, for 
example. It also goes without saying that an above- 
described spherical menu system may also be formed from 
35 ellipsoidal or even polygonal three-dimensional forms. 

The inventive virtual three-dimensional user interface 
is described in greater detail below with reference to 
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exemplary embodiments which are illustrated in the 
drawing and which reveal further features and 
advantages of the invention. In said exemplary 
embodiments, identical or functionally identical 
5 features are referenced using corresponding reference 
symbols . 



In the drawing: 



10 fig. 1 shows a simplified overview of an 

immersive VR (virtual reality) . graphics 
system (which is concerned in this case) 
according to the prior art; 

15 figs. 2a, b show two diagrammatically illustrated 

exemplary embodiments of the inventive 
virtual three-dimensional user 

interface; 

20 figs. 3a-c show a perspective view of a preferred 

exemplary embodiment of the inventive 
user interface (in this case: spherical 
menu) for use in a VR graphics system 
(fig. 2a) shown in fig. 1 and two 

2 5 typical interaction sequences (figs. 3b 

and 3c) using the user interface shown 
in fig. 3a; 

fig. 4 uses a flowchart to show a typical 

3 0 functional sequence when controlling an 

inventive user interface; and 

fig. 5 shows a functional sequence, which is 

more detailed in comparison with fig. 4, 
3 5 in the user interface shown in figures 

3a - 3c. 
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The VR graphics system which is diagrammatically 
illustrated in fig. 1 has a projection screen 100 in 
front of which a person (user) 105 stands in order to 
view the scene 115, which is generated there via a 
5 projector 110, using stereoscopic glasses 120. It goes 
without saying that auto-stereoscopic screens or the 
like may also be used in the present case instead of 
the stereoscopic glasses 120. In addition, the 
projection screen 100, the projector 110 and the 
10 glasses 12 0 may be replaced in the present case with a 
data helmet which is known per se and then comprises 
all three functions. 

The user 105 holds an interaction unit 125 in his hand 
15 in order to generate preferably absolute position data 
such as the spatial position and orientation of the 
interaction unit in the physical space and to transmit 
said data to a position detection sensor system 130 - 
140. Alternatively, however, relative or differential 
20 position data may also be used but this is not 
important in the present context . 

The interaction unit 125 comprises a position detection 
system 145, preferably an arrangement of optical 

25 measurement systems 145, both the absolute values of 
the three possible angles of rotation and the absolute 
values of the translational movements of the 
interaction unit 125, which are possible in the three 
spatial directions, being detected using said 

3 0 arrangement of measurement systems and being processed 
in real time by a digital computer 150 in the manner 
described below. Alternatively, these position data may 
be detected using acceleration sensors, gyroscopes or 
the like which then generally provide only relative or 

35 differential position data. Since this sensor system is 
not important in the present case, a more detailed 
description is dispensed with here and reference is 
made to the documents mentioned at the outset. 
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Said absolute position data are generated by a computer 
system which is connected to the interaction unit 125. 
To this end, they are transmitted to a microprocessor 
5 160 of a digital computer 150 in which, inter alia, the 
necessary graphical evaluation processes (which are to 
be assumed to be familiar to a person skilled in the 
art) are carried out in order to generate the 
stereoscopic three-dimensional scene 115. The three- 
10 dimensional scene representation 115 is used, in 
particular, for visualizing object manipulations, for 
three-dimensional navigation in the entire scene and 
for displaying function selection structures and/or 
menu structures. 

15 

In the present exemplary embodiment, the interaction 
unit 125 is connected, for carrying data, to the 
digital computer 15 0, via a radio connection 17 0, using 
a reception part 165 (which is arranged there) . The 
2 0 position data which are transmitted from the sensors 
145 to the position detection sensor system 130 - 140 
are likewise transmitted in a wireless manner by radio 
links 175 - 185. 

2 5 Additionally depicted are the head position (HP) of the 

user 105 and his viewing direction (VD) 190 with 
respect to the projection screen 100 and the scene 115 
projected there. These two variables are important for 
calculating a current stereoscopic projection insofar 

3 0 as they considerably concomitantly determine the 

necessary scene perspective since the perspective also 
depends, in a manner known per se, on these two 
variables . 

3 5 In the present exemplary embodiment, the interaction 
unit 12 5 comprises a pushbutton 195 which the user 105 
can use, in addition to said possibilities for moving 
the interaction unit 12 5 in the space, to trigger a 
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particular interaction, as described below with 
reference to fig. 3. It goes without saying that two or 
more pushbuttons may also alternatively be arranged in 
order to enable further different interactions, if 
5 appropriate. Instead of one or more pushbuttons, 
corresponding user inputs may also be effected, as 
already mentioned, using voice, gestures or the like. 
Additionally depicted in fig. 1 are the head position 
(HP) of the user 105 and his viewing direction (VD) 190 

10 with respect to the projection screen 100 and the scene 
115 projected there. These two variables are important 
for calculating a current stereoscopic projection 
insofar as they considerably concomitantly determine 
the necessary scene perspective in a manner known per 

15 se . 

The central element of the immersive VR graphics system 
shown is the stereoscopic representation (which is 
guided (tracked) using the position detection sensor 

20 system 130 - 140) of the respective three-dimensional 
scene data 115. In this case, the perspective of the 
scene representation depends on the observer's vantage 
point and on the head position (HP) and viewing 
direction (VD) . To this end, the head position (HP) is 

25 continuously measured using a three-dimensional 
position measurement system (not illustrated here) and 
the geometry of the view volumes for both eyes is 
adapted according to these position values. This 
position measurement system comprises a similar sensor 

30 system to said position detection system 130 - 140 and 
may be integrated in the latter, if appropriate. A 
separate image from the respective perspective is 
calculated for each eye. The difference (disparity) 
gives rise to the stereoscopic perception of depth. 

35 

In the present case, an interaction by a user is 
understood as meaning any action by the user, 
preferably using said interaction unit 125. Included in 
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this case are the movement of the interaction unit 12 5 
as shown in figures 2a, 2b and 3a - 3c and the 
operation of one or more pushbuttons 195 which are 
arranged on the interaction unit 125. Acoustic actions 
5 by the user, for example a voice input, or an action 
determined by gestures may additionally be included. 

Figures 2a and 2b show two exemplary embodiments (which 
are illustrated only diagrammatical ly in this case) of 
10 the inventive virtual three-dimensional user interface, 
said exemplary embodiments being intended to be used to 
explain only the fundamental method of operation of the 
inventive user interface. 

15 In the exemplary embodiment shown in fig. 2a, provision 
is made of two subelements 250 and 255 which are 
approximately square. It goes without saying that this 
diagrammatically highly simplified illustration may be 
used only to illustrate the fundamental technical 

2 0 concepts and, in the present field of use of the VR 
graphics systems, these subelements will likewise be 
preferably three-dimensional, for example in the form 
of three-dimensional cubes, cuboids, spheres or the 
like. In the present example, each of the two 

2 5 subelements 2 50, 2 55 has four action elements, to be 

precise the respective outer sides of the two squares, 
which are available for interactions by a user, in 
particular. Two of these outer sides 260, 265 are 
respectively emphasized in this illustration using a 

3 0 double line. The dashed arrows 271 and 272 are intended 

to indicate that, in the interaction shown here, the 
subelement 250 is rotated 271 and displaced 272 in such 
a manner that the outer side 2 60 comes to rest on the 
outer side 265. This second interaction phase is 
35 illustrated in the lower half of fig. 2a. 

Combining the two subelements 25 0, 2 55 on the two outer 
sides 260, 265 now triggers an action or function which 
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will be described in even more detail below. The action 
or function is triggered, in particular, when the outer 
sides 260, 265 have reached a particular degree of 
convergence or only when they have come into contact 
5 virtually (i.e. in the current VR scene). It goes 
without saying that further actions or functions may 
also be triggered by the other possible interactions 
between the remaining outer sides of the subelements 
250, 255. 

10 

In the exemplary embodiment shown in fig. 2b, one 
subelement is again square 270, whereas the second 
subelement is formed by a ring 275 which is arranged 
concentrically around the square subelement 27 0 in the 
15 initial position of the latter. In the present example, 
the ring 275 is subdivided into four segments 275, each 
of these segments 27 5 being assigned to a separate 
action or function. 

2 0 In this exemplary embodiment, interactions are carried 
out by the square subelement 27 0 first of all being 
rotated into a new position (i.e. spatial orientation) 
280. The subelement 270 is then displaced, by means of 
a trans lational movement that corresponds to the two 

2 5 movement paths 2 90 which are shown only by way of 
example here, either into the position designated 1 1' 
(in the round circle) or into the position which is 
correspondingly designated ' 2 ' . In the example of ' 1 1 , 
the square subelement 27 0, 280 comes into (virtual) 

30 contact, at the respective corners 295, 295', with the 
ring segment shown, as a result of which an action or 
function is triggered. In the other case of '2', the 
action or function is triggered only when there is an 
overlap 285 (shown) between the two subelements 270, 

35 275. 



It should be noted that, in an alternative refinement, 
the two subelements shown may also both be moved, for 
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example toward one another. That is to say it is only 
the relative movement (shown) between the two 
subelements which is important in the present case. The 
two subelements may also even be controlled by a user 
5 with both hands, the user then holding a separate 
above-described interaction unit in each hand. 

Fig. 3a shows, in a perspective illustration, a view of 
a preferred exemplary embodiment of the inventive user 
10 interface, to be precise using the example of a 
spherical menu system (which was already described at 
the outset) in the present case. Figures 3b and 3c 
which are described below show two typical interaction 
sequences for this spherical menu system. 

15 

In the exemplary embodiment, it shall be assumed that 
the interaction unit has two buttons 195 (fig. 1) which 
can preferably be operated using the user's thumb and 
index finger. These two buttons may be used in two 
2 0 ways, to be precise by briefly pushing them (clicking) 
and by holding them down (holding) for a relatively 
long time. These two actions result in a total of four 
possible interactions, i.e. in the present exemplary 
embodiment: clicking using the thumb = termination, 

2 5 clicking using the index finger = action, holding using 

the thumb = gripping and holding using the index finger 
= menu. A graphical menu system is thus gripped by 
holding the button assigned to the thumb and the menu 
is held by holding down the thumb button. Menu systems 

3 0 which have already been inserted into the scene may be 

activated by pushing and holding down the index finger 
button. A function is left or terminated by clicking 
the thumb button. 

3 5 As can be seen from fig. 3a, the present spherical menu 
system has three parts and comprises an inner menu 
sphere 200 which provides a status indication (already 
mentioned) for indicating the instantaneous state of 
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the menu system as regards the menu hierarchy. The 
inner menu sphere 2 00 is surrounded by a spherical 
shell 2 05 which has four parts in the present case and, 
by means of rotation, can be used to activate different 
5 menu entries in the main menu ("main")/ to be precise 
the entries "group", "snap", "state" and "meas" in the 
present case. A further four menu entries "work", 
"single", "fly" and "extra" are situated in four 
corresponding segments 210 of an outer menu ring. 

10 

The main menu ("main") of the spherical menu system 200 
- 210, which has three parts in the present case, 
therefore contains eight different menu entries, four 
of which can be reached by rotating the interaction 
15 unit 12 5 (not shown here) and four of which can be 
reached by translating (displacing) the interaction 
unit 125. 

When the interaction unit 12 5 is physically rotated in 
2 0 a three-dimensional manner on account of a 
corresponding hand movement by the user, the inner menu 
sphere 2 00 rotates in a corresponding manner. As of an 
angle of rotation of 60° in the present case, the inner 
menu sphere 2 00 latches with an orientation which has 

2 5 been displaced through 90° relative to the previous 

orientation, i.e. the "play" of the latching function 
is 3 0° in the present case. In the example, this 
latching is activated by releasing one of the two 
buttons of the interaction unit 125. 

30 

Displacing (translating) the inner sphere 200 along one 
of four possible translation paths, which are 
prescribed using "elastic-band-like" guides 215, makes 
it possible for the inner sphere 2 00 to be moved to the 

3 5 four menu entries 210 which are arranged in the form of 

a ring. The function corresponding to the respective 
menu entry 210 is activated when the inner sphere 200 
touches or overlaps (fig. 2c) the respective ring 
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segment 210. This makes it possible to rapidly change 
over between different menus or submenus. 

Figures 3b and 3c now illustrate typical interaction 
5 sequences, which are respectively designated using 
sequences of digits 1. - 3., when a function is 
selected by rotating the spherical shell 2 05 and 
displacing the spherical shell 205 in a translational 
manner . 

10 

In the case of a pure rotation (fig. 3b) , one of the 
two buttons 195 of the interaction unit 12 5, which are 
present in this case, is first of all pushed and is 
then held down, as a result of which the menu system is 
15 first of all inserted into the current scene. The 
interaction unit 12 5 is then physically rotated by the 
user. As of a threshold value for the physical rotation 
of the interaction unit (30° in the present example) , 
the inner sphere 2 00 also begins to rotate in a 

2 0 corresponding manner. Releasing the button 195 causes 

the inner sphere 2 00 and the spherical shell 2 05 to 
latch at the respective nearest latching point in the 
30° gradation, the latching function "snap" thus being 
selected in the present example. 

25 

In the case of a pure displacement (fig. 3c) , the 
button 195 is again pushed and held down, as a result 
of which the menu system first of all appears in the 
scene. Physically moving the interaction unit 125 in a 

3 0 translational manner causes the inner sphere 2 00, 

together with the spherical shell 2 05, to be displaced 
until it comes to touch or overlap one of the ring 
segments 210. As soon as this touching occurs, a new 
menu or submenu appears or a prescribed function 
3 5 selection takes place. Even the translational movement 
results in the change in shape (shown) of the spherical 
shell. The spherical shell 205 may also be animated in 
a corresponding manner in the case of the sphere or 
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spherical shell touching or overlapping the respective 
ring segment 210 affected. 

The present exemplary embodiment also provides that, as 
5 of a particular prescribable partial overlap or partial 
touching between the spherical shell 2 05 or the sphere 
200 and the respective ring segment 210 affected, a 
further relative displacement between these two 
subelements is suppressed, which approximately 
10 corresponds to translation-related latching. 

One variant may provide for the overlapping-dependent 
interaction which is shown in figures 2b and 3c to be 
activated, when a menu entry associated with the 

15 spherical shell 2 05 and a menu entry associated with 
the ring 210 overlap (fig. 3c), in such a manner that 
the menu entries which are combined in the process 
simultaneously carry out a logic (boolean) combination. 
Provision may thus be made, for instance, for the 

2 0 "snap" function shown in fig. 3b to activate either a 
combined " single- snap" or 11 extra -snap" function when it 
touches or overlaps one of the ring segments 210. It 
goes without saying that other types of combination 
such as 'OR' or 'NOT' may also be provided instead of 

2 5 such an 'AND' combination. 

Fig. 4 now uses a flowchart to show a typical 
functional sequence when controlling the inventive user 
interface. After the start 3 00 of the routine, a check 

30 is first of all carried out 305 in a loop in order to 
determine whether a particular pushbutton of the 
interaction unit 12 5 has been operated. If this is the 
case, the two subelements 2 50, 2 55 and 27 0, 27 5 shown 
in figures 2a and 2b are inserted into the current VR 

35 scene 310. Otherwise, step 305 is carried out again, if 
appropriate after a certain delay, in accordance with 
said loop. 
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A check is then carried out in step 315 in order to 
determine whether a virtual movement (which is caused 
by physical movement of the interaction unit) of at 
least one of the two subelements 250, 255, 270, 275 has 
5 been effected in the scene. If no movement at all of 
either of the two subelements has been determined, the 
process jumps back to step 310. Otherwise, in 
accordance with such a movement, the respective 
subelement is preferably displayed in animated form in 

10 the scene 320 in order to also visually indicate the 
movement (rotation or translation) . A check is carried 
out in the following step 32 5 in order to determine 
whether the two subelements 250, 255, 270, 275 have 
come into physical contact (or spatial proximity) 

15 during the movement. If this is not the case, the 
process jumps back to step 310 in order to carry out 
the steps which have just been described again. 
Otherwise, a particular action or function is triggered 
330 on the basis of the respective contact region or 

2 0 the respective ring segments 275 involved. Finally, a 
check is carried out in step 33 5 in order to determine 
whether the function triggered in step 330 is a 
function that terminates the entire procedure. 
Alternatively, a check can be carried out in this case 

2 5 in order to determine whether said pushbutton was 

operated again for said purpose of terminating the 
procedure. If this is the case, the procedure is 
terminated in step 340. Otherwise, the process jumps 
back to step 310 in order to carry out the above- 

3 0 described steps again. 

Fig. 5 finally shows a typical functional sequence of 
the exemplary embodiment (illustrated in figures 3a - 
3c) of the inventive user interface. After the start 
35 400 of the routine shown, a check is first of all 
carried out 405 in the form of a loop in order to 
determine whether a (particular, if appropriate) 
pushbutton 195 of the interaction unit 12 5 (said thumb 
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button in the present case) has been operated by the 
user. If it is determined that the pushbutton has been 
operated, the loop is left and the above-described 
spherical menu system 200 - 210 is inserted in the 
5 present case into the scene in the next step 410. A 
check is then carried out 415 in order to determine 
whether the interaction unit 12 5 has undergone user- 
guided rotation. If this condition 415 applies, a check 
is then also carried out 42 0 in order to determine 

10 whether, a prescribable threshold value for the rotation 
(3 0° in the present case) was exceeded during the user- 
guided rotation of the interaction unit 125. If this 
condition 420 has also been satisfied, the spherical 
shell is latched 425, both functionally and in 

15 visualized form, into the respective new angular 
orientation. 

If no rotational movement of the interaction unit was 
detected 415 or if, in the case of detected rotation, 

2 0 said threshold value was not exceeded 42 0, the process 
changes to step 43 0 in which a check is also carried 
out in order to determine whether the interaction unit 
125 has been displaced (in a translational manner) . If 
this condition 43 0 applies, a sphere 2 00 or spherical 

25 shell 2 05 which has been correspondingly displaced is 
animated 43 5 in the scene, for example in the manner 
shown in fig. 2c. A check is then carried out 440 in 
order to determine whether the inner menu sphere 2 00 
has touched or overlapped one of the outer ring 

30 segments 210. If this is the case, a function or menu 
selection assigned to the overlapped ring segment 210 
is activated or triggered in step 445. 

However, if the condition 430 has not been satisfied or 
35 if, in the event of this condition 430 having been 
satisfied, the condition 440 has not been satisfied, 
the process jumps to step 450 in which a check is 
carried out in order to determine whether the function 
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triggered in step 445 is a function that terminates the 
entire procedure with a view to removing the spherical 
menu system from the current scene again. 
Alternatively, a check may be carried out in this case 
5 in order to determine whether the pushbutton has been 
operated again or the like. If this condition 450 
finally applies, the routine is terminated 455 or the 
process jumps back to step 415 in order to detect an 
interaction again in the manner described above. 



10 
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Patent claims 



1. A graphical user interface for controlling a 
5 virtual reality (VR) graphics system by means of 

interactions with a function selection system that 
provides at least two functions, the VR graphics 
system having a projection device for visualizing 
a virtual three-dimensional scene and the 

10 interactions with the VR graphics system being 

effected using at least one interaction unit 
which, in interaction with a position detection 
sensor system for detecting a respective physical 
spatial position and/or orientation of the 

15 interaction unit, is used to provide position data 

in the VR graphics system, characterized by an 
interaction element which is functionally and 
visually formed from at least two subelements 
which respectively provide said function 

2 0 selection, the at least two subelements being 

designed such that they can be moved in a virtual 
three-dimensional manner relative to one another 
by means of a physical three-dimensional movement 
of the interaction unit, and said function being 

2 5 selected by means of the at least two subelements 

being moved in a virtual three-dimensional manner 
relative to one another. 

2. The user interface as claimed in claim 1, 

3 0 characterized in that at least one of the 

subelements is at least occasionally displayed at 
an essentially fixed position in the virtual 
scene, said function being selected by means of a 
virtual three-dimensional movement of the 
3 5 respective other subelement relative to the 

subelement which is at least occasionally 
displayed at the fixed position. 
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3. The user interface as claimed in claim 1 or 2 , 
characterized in that the function selection is 
triggered, during the movement of the at least two 
subelements relative to one another, if the at 

5 least two subelements at least partially touch or 

overlap . 

4 . The user interface as claimed in one of claims 1 
to 3, characterized in that the at least two-part 

10 interaction element is implemented in the form of 

a menu system, a function selection system or the 
like . 

5. The user interface as claimed in claim 4, 
15 characterized in that the interaction element is 

formed by a spherical menu system which comprises 
three visual subelements and comprises an inner 
sphere which is formed in one part, a spherical 
shell which is formed from at least two spherical 
20 shell segments and is arranged on the visual 

surface of the inner sphere and a ring which is 
arranged in the outer region of the sphere or 
spherical shell and comprises at least two ring 
segments, the inner sphere providing to represent 

2 5 an item of state information relating to the 

instantaneous state of the spherical menu system. 

6. The user interface as claimed in claim 5, 
characterized in that the state information 

3 0 indicates the menu level which is currently 

activated in the spherical shell segments in 
accordance with a menu tree. 

7. The user interface as claimed in claim 5 or 6, 
3 5 characterized in that the spherical shell segments 

can be correspondingly rotated about the inner 
sphere, by means of user-guided rotation of the 
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interaction unit, in order to make it possible to 
activate various spherical shell segments. 



8. The user interface as claimed in one of the 
5 preceding claims, characterized in that provision 

is made of a latching function which depends on 
the angle of rotation of the interaction unit 
and/or of the respective subelement. 

The user interface as claimed in one of the 
preceding claims, characterized in that an 
interaction which is to be effected on the basis 
of a physical rotational movement and/or physical 
translational movement of the interaction unit is 
triggered a corresponding interaction only when an 
empirically prescribable threshold value is 
exceeded. 

10. The user interface as claimed in one of claims 3 
20 to 9, characterized in that a further functional 

and/or visual relative displacement between the 
subelements is prevented as of a prescribable 
degree of overlap or touching between the at least 
two subelements. 

25 

11. The user interface as claimed in one of the 
preceding claims, characterized in that the 
relative displacement between the at least two 
subelements is effected in a guided manner. 

30 

12 . The user interface as claimed in one of the 
preceding claims, characterized in that at least 
one of the subelements is visually displayed in 
animated form in the scene in the event of 
35 rotation and/or translation and/or touching. 



10 9. 



15 



The user interface as claimed in one of the 
preceding claims, characterized in that the 
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interaction unit has at least one control element 
which is used to at least assist said functional 
sequences of the user interface. 

14 The user interface as claimed in one of the 
preceding claims, characterized in that sa.d 
functional sequences of the user interface are 
assisted using voice input and/or the detection of 
gestures or facial expressions of the user. 

15 The user interface as claimed in one of the 
preceding claims, characterized in that saxd 
touching or overlapping function comprises at 
least one logic operation. 

16 A virtual reality (VR) graphics system having a 
graphical user interface as claimed in one of the 
preceding claims. 
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Fig. 2b 
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Visual insertion of the two subelements into the current scene 




Animated display of the movement of the 
at least one subelement in the scene 



Triggering of a function on the basis 
of the respective contact region 
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Fig. 4 
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Fig. 5 



Latching of the spherical 
shell in the rotated orientation 



Animated display of a displaced spherical 
shell (n sphere) of the menu system 
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440 



Triggering of a function assigned to the ring 
segment which has been touched/overlapped 
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450 



455 



